[10/N] test(zh): add objective Chinese regression guards / 新增客观中文回归 CI 守卫 - #841
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
edwingao28
changed the title
edwingao28
changed the title
edwingao28
changed the title
|
Claude finished @edwingao28's task in 4m 9s —— View job Re-review:
|
| Test | Reported violation | Looks like |
|---|---|---|
i18n.test.ts |
route-sibling missing zh for /model, /model/[slug] |
Genuine — new routes with no /zh sibling (or need explicit exemption) |
blog-content.test.ts (jalapeño) |
protected-token missingFromZh=["MW"×8] missingFromEn=["tok/s/user"×3] |
False positive — bare unit MW in prose ("throughput per MW") is legitimately rendered into Chinese; the identical-multiset check over-fires |
blog-content.test.ts (agentx) |
inline-code + protected-token --enable-prefill-delayer |
EN writes the flag as bare prose, ZH wraps it in ` backticks → real EN/ZH inconsistency the guard can't reconcile |
zh-copy.test.ts |
chip-untranslated ×2 in jalapeño zh |
False positive — fires on the proper noun "Hot Chips" / English URLs / "chiplet", not an untranslated common-noun "chip" |
Why it matters: the PR cannot merge with a red required check, and the failures show the guard currently flags legitimately-translated content. Because these tests scan all repo content rather than the PR diff, this guard will also fail CI on unrelated future PRs whenever a post uses a bare unit word in prose or a "chip" proper noun.
Fix: rebase onto master, then for each case either (a) tighten the rule so bare units in prose and proper-noun "chip"/"Hot Chips" don't count, (b) add a documented entry to zh-objective-guard-exceptions.json, or (c) bring the content into compliance (add the /model zh pages or exempt them; make EN/ZH backtick the flag consistently). The MW/tok/s/user and "Hot Chips" cases specifically read as guard over-reach — the kind of context-dependent terminology the PR description says stays out of deterministic CI.
🟡 WARNING — ReDoS in the link-reference regex (zh-objective-guard.ts:563)
CodeQL alert #25 is a true positive. (?:\\.|[^\]\n])+ lets a backslash match two ways, giving exponential backtracking (measured 12.8s at 30 chars). Trusted build-time input so not exploitable, but a malformed line could hang CI. One-char fix suggested inline ([^\]\n] → [^\]\n\\]), verified equivalent and blow-up-free.
🟢 CodeQL alert #26 (:766) — false positive
Tag-stripping there feeds a Han-character lint scan, never HTML output; no XSS surface. Noted inline; optional to dismiss.
The guard machinery itself is careful, well-tested in isolation, and the design intent (objective-only invariants) is sound. The blocker is purely that it's currently red against live master content. Once rebased and the false positives are tightened/excepted, this should be good.
· branch feat/zh-objective-ci-guard
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Add mutation-tested route, Blog, dictionary, protected-token, structured-data, and explicit Chinese-only English-byte guards. Document exact temporary exceptions and keep editorial judgment out of deterministic CI. 中文：新增基于 mutation 的路由、Blog、字典、受保护 token、结构化数据及显式中文专属英文逐字节检查。记录精确的临时例外，并确保确定性 CI 不介入编辑性判断。
Make the Chinese-only CLI path- and rename-aware, pair locale dictionaries per object, and extend Blog protection to real MDX links, CommonMark delimiters, and bidirectional protected-token multisets. Scan visible JSX/MDX prose independently and document exact temporary baseline exceptions. 中文：强化客观中文本地化检查。修复仅中文模式的路径与重命名处理，按对象核对中英文字典，并覆盖真实 MDX 链接、CommonMark 分隔符和双向受保护 token 多重集；同时独立扫描 JSX/MDX 可见文案，并记录精确、可删除的临时基线例外。
Anchor protected-token waivers to complete Blog pairs, parse complete Markdown and whitespace-tolerant JSON-LD structures, preserve JSON paths, and cover static template props, shorthand dictionary keys, real unit forms, and Git type changes. 中文：补齐客观中文检查的解析边界。将受保护 token 豁免绑定到完整中英文文章对，完整解析 Markdown 链接与允许空白的 JSON-LD 结构并保留 JSON 路径，同时覆盖静态模板属性、字典 shorthand key、真实单位形式和 Git 类型变更。
Recognize slash and hyphen GPU/chip-hour forms, keep static MDX link props intact across JSX comparison expressions, and prevent overlapping protected blocks from leaking into prose scans. Add consumptive, pair-pinned exceptions for the six existing JSON-LD translations.\n\n中文：保护 GPU-hour 文案不变量。兼容 GPU/chip-hour 的斜杠与连字符写法，避免 JSX 比较表达式截断 MDX 静态链接属性，并防止重叠保护区间泄漏到正文扫描；同时为现有六处 JSON-LD 译文加入按次消费、绑定文章对摘要的临时例外。
Protect slash and hyphen GPU/chip-hour forms inside JSON-LD, normalize GPU-hour cost rates before casing, and keep MDX tags intact across braces in JavaScript comments and regex literals. Refresh only the exact pair-pinned baseline exceptions made necessary by the expanded matcher. 中文：补全受保护单位解析。JSON-LD 中的 GPU/chip-hour 斜杠与连字符写法现均受保护；成本单位会先归一化 GPU-hour 再处理大小写；MDX 标签解析会正确跳过 JavaScript 注释和正则表达式中的大括号。基线例外仅按扩展 matcher 后的实际差异更新，并继续绑定完整文章对摘要。
Replace the heuristic MDX expression lexer with Acorn tokenization so division, regex literals after spread, comments, and nested template interpolation cannot truncate static link props. Add direct mutation coverage for each lexical boundary. 中文：使用 Acorn token 解析 MDX 属性表达式，替换原有启发式 lexer，避免除法、spread 后的正则表达式、注释和嵌套 template interpolation 截断静态链接属性；并为每类词法边界补充直接 mutation 覆盖。
Extend the objective guard tokenizer with JSX support so nested elements, fragments, arrow-returned JSX, and nested JSX expressions cannot hide or falsely expose static link props. Add accepted and bad-link mutations for every compiler-valid form. 中文：为客观检查的 tokenizer 增加 JSX 支持，避免嵌套元素、fragment、箭头函数返回的 JSX 及 JSX 内部表达式隐藏或误暴露静态链接属性；并为每种可编译写法补充通过与错误链接 mutation。
3cc309d to
4393f7a
Compare
|
Claude finished @edwingao28's task in 2m 43s —— View job Re-review:
|
| Test | Violation | Assessment |
|---|---|---|
i18n.test.ts |
route-sibling × 2 — /model, /model/[slug] missing zh |
Genuine — new routes on master, need /zh siblings or an explicit exemption |
blog-content.test.ts (jalapeño) |
protected-token missingFromZh=["MW"×8] missingFromEn=["tok/s/user"×3] |
Guard over-reach — bare units in prose; the multiset check over-fires |
blog-content.test.ts (agentx) |
inline-code + protected-token --enable-prefill-delayer |
EN writes the flag as bare prose, ZH backticks it → real EN/ZH inconsistency |
zh-copy.test.ts |
dictionary-key-parity — CollectiveXDisplay.tsx:67, ~70 keys missingFromEn |
Fires on a component that landed on master; en/zh STRINGS dicts diverged (or guard over-reach) |
zh-copy.test.ts |
recorded exception fingerprint baf66a66… no longer matches |
Stale exception — the CollectiveX component changed on master, so the pinned SHA-256 drifted |
zh-copy.test.ts |
chip-untranslated × 2 in jalapeño zh |
Guard over-reach — fires on "Hot Chips" (conference proper noun) and "chiplet", not an untranslated common-noun "chip" |
Why it matters: the PR cannot merge with a red required check, and several failures show the guard flagging legitimately-translated content — the bare-unit MW/tok/s/user and the "Hot Chips"/"chiplet" cases are exactly the context-dependent terminology the PR description says stays out of deterministic CI. Because these tests scan all repo content (not the PR diff), the guard will also break CI on unrelated future PRs whenever a post uses a bare unit word in prose or a "chip" proper noun, and every change to master can invalidate a pinned exception fingerprint (as just happened to CollectiveX).
Fix: rebase onto master, then per case either (a) tighten the rule so bare prose units and proper-noun "chip"/"Hot Chips"/"chiplet" don't count, (b) add a documented entry to zh-objective-guard-exceptions.json, or (c) bring the content into compliance (/zh/model pages or exempt them; backtick the flag consistently in EN/ZH; reconcile the CollectiveX en/zh dict + refresh its exception fingerprint). See inline comments on i18n.test.ts:42 and zh-objective-guard.ts:620.
🟡 WARNING — ReDoS at zh-objective-guard.ts:563 still unfixed
CodeQL alert #25 is a true positive and the one-char fix from the previous review was not applied in any of the 7 commits. (?:\\.|[^\]\n])+ lets a backslash parse two ways → exponential backtracking (~12.8s at 30 chars). Build-time/trusted input so not exploitable, but a malformed line hangs CI. Suggestion posted inline ([^\]\n] → [^\]\n\\]).
🟢 CodeQL alert #26 (:766) — false positive
Tag-stripping there feeds a Han-character lint scan, never HTML output — no XSS surface. Optional to dismiss. (Unchanged from prior review.)
The guard machinery is careful and well-tested in isolation, and the objective-only design intent is sound. But the blocker from the last review persists and has grown: the guard is red against live master, and part of that redness is the guard enforcing the very editorial judgments (bare units, proper-noun "chip", per-post flag formatting) the PR says it excludes. No browser surface, so no runtime verification needed here — the blocker is CI, not behavior.
· branch feat/zh-objective-ci-guard
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.
There are 3 total unresolved issues (including 1 from previous review).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 4393f7a. Configure here.
…en scanner - Exempt intentionally English-only /model and /model/[slug] routes in the route-sibling check (their metadata declares canonical without hreflang). - Refresh the CollectiveXDisplay dictionary-parity fingerprint after the master-side refactor (74 -> 73 Chinese-only keys). - Add exact blog exceptions for the ported CUDA-moat and Jalapeño articles (inline-code formatting drift and MW/tok-s-user protected-token drift owned by the content branches). - Stop scanning markdown link destinations for untranslated terms; URL slugs like .../inference-chip/ are not translatable copy. - Fix CodeQL findings: remove regex alternation overlap in the reference-link pattern (exponential backtracking) and strip MDX tags to a fixed point (incomplete multi-character sanitization).
|
Claude finished @edwingao28's task in 3m 6s —— View job Re-review:
|
| Prior finding | Status | How |
|---|---|---|
ReDoS in link-reference regex (zh-objective-guard.ts:563, CodeQL #25) |
✅ Fixed | Backslash excluded from the negated class — [^\]\n] → [^\\\]\n], so each \ has exactly one parse. |
chip-untranslated false-positive on URL slugs / "Hot Chips" / "chiplet" |
✅ Fixed | New withoutMarkdownLinkDestinations strips ](url) before the Han scan; fixed-point tag-strip loop added. Covered by a new test. |
route-sibling — /model, /model/[slug] missing zh |
✅ Fixed | Documented EN_ONLY_ROUTES exemption. /model is intentionally English-only (alternates.canonical without hreflang, model/page.tsx:20). The test guards the guard: it asserts hasZhSibling(route) === false, so the exemption auto-breaks if a /zh/model sibling is ever registered. |
protected-token MW / tok/s/user (jalapeño); --enable-prefill-delayer (agentx) |
✅ Handled | Documented per-pair exceptions, gated by a tamper-evident pairSha256 pin so any edit to the article pair re-arms the guard. |
dictionary-key-parity + stale exception fingerprint (CollectiveX) |
✅ Fixed | Fingerprint refreshed to the current 73-key mismatch (8bcba14…). |
The ReDoS and CodeQL-#26 (Han-lint tag-strip, confirmed a false positive) alerts are both addressed/benign. New code (withoutMarkdownLinkDestinations, the guard-of-the-guard) ships with tests.
🟡 Non-blocking — latent over-reach to keep an eye on
Not firing today (whole-repo scan is green), so not blocking, and already noted by Cursor Bugbot on an earlier commit — flagging only so it isn't lost:
chineseSegmentswhole-file fallback (zh-objective-guard.ts~L805): for a non-MDX file whose only Han sits in a code comment (no string/template/JSX node), the AST walk finds no segments and falls back to[source], scanning English identifiers too. A future.ts/.tsxwith a Chinese comment plus aChipidentifier could then false-positivechip-untranslated. Consider stripping comments (or returning[]) instead of falling back to the raw source.
Note
Branch is 1 commit behind master (#836, which also touches tests-unit.yml / package.json / docs/chinese-copy.md). CI already validated the PR-merged-with-master result as green, so the guard passes against current master content — a final rebase before merge is advisable only to keep the diff clean.
🟢 Re-review verdict: LGTM — all previously-blocking failures are fixed, CI is green, and the new scanner logic is tested. The one remaining item is a pre-existing latent edge case, not a regression this branch introduces.
· branch feat/zh-objective-ci-guard
中文：简化客观中文文案检查，移除冗余规则与例外，并同步最新 master。
|
Claude finished @edwingao28's task in 2m 29s —— View job Re-review:
|
| Prior concern | How the rewrite handles it |
|---|---|
| ReDoS in the link-reference-definition regex (CodeQL #25) | The hand-rolled regex MDX scanner is gone. zh-objective-guard.ts now parses MDX through @mdx-js/mdx + remark-gfm/remark-math and walks the AST — there is no vulnerable backtracking regex left, and acorn/acorn-jsx are dropped. |
chip-untranslated over-reach (Hot Chips / chiplet / URL slugs) |
findMechanicalCopyViolations and its whole "shared conservative scanner" describe block are removed from zh-copy.test.ts. That editorial rule is no longer in deterministic CI. |
protected-token bare-unit over-fire (MW, tok/s/user) |
The protectedTokens multiset check is removed. Inline-code parity is now one-directional (missingMultisetValues(en, zh)), so Chinese prose may add backtick formatting without dropping an English identifier — no bare-unit comparison remains. |
Stale exception fingerprints / zh-objective-guard-exceptions.json drift |
The exceptions file is deleted entirely; the remaining checks are exact-by-construction, so there is nothing left to re-pin against master. |
Unconditional fetch-depth: 0 (Cursor) |
Removed from the main vitest job; the deep checkout now lives only in a separate zh-copy-only job gated by the chinese-copy-only label. |
Route siblings / /model |
findRoutePairViolations now takes a direction-separated { englishOnly, chineseOnly } waiver; /model now has real /zh/model siblings and the only exception is the documented Chinese-only /[...notFound] catch-all. |
What I checked:
- CI:
Tests (Unit)✅,Lint & Format✅,Zizmor✅.Tests (E2E)/ Code Quality still running, but this branch changes no browser behavior (test/lib/CI only), so nothing to verify at runtime — no Playwright pass applies. - Dependencies:
@mdx-js/mdxadded as a devDependency;remark-gfm/remark-mathalready declared. The guard is imported only by tests andscripts/check-zh-chinese-only.ts(no app-runtime import), so a devDep is correct. - New behavior is correct: MDX parse errors are caught and surfaced as a clean
mdx-syntaxviolation instead of hanging; links, figure sources, math, fenced/inline code, and JSON-LD shape/protected-values are collected from the AST symmetrically for both locales. - Test coverage:
zh-objective-guard.test.tsexercises the new AST paths directly (route parity both directions + one-sided waivers, nested dictionary mismatch, shorthand/computed/spread handling, English-byte preservation with swapped initializers, translated-prose acceptance, additive inline-code, JSON-LD path binding, malformed-MDX handling) plus git-fixture CLI integration for rename/copy/delete. Good mutation-style coverage.
No blocking or warning issues in the new changes.
· branch feat/zh-objective-ci-guard
2 participants
Part of #823 / 属于 #823 路线图的一部分
What changed
en/zhdictionary structure, and Chinese-only English-byte preservation./zhlinks.Validation
Review notes
中文说明
en／zh字典结构，以及中文专属改动中的英文字节保护。/zh链接。验证
审核说明
Note
Medium Risk
Adds broad CI gates on routes, bilingual dictionaries, and blog MDX that can block merges on structural drift; the labeled English-byte guard affects PR workflow but does not touch runtime or auth.
Overview
Introduces
zh-objective-guardas the shared enforcement layer for bilingual invariants: direction-aware App Router/zhpage pairing (with/zh/[...notFound]as the documented exception), TypeScript AST checks that expliciten/zhobject literals share the same key shape, and MDX-aware blog pairing for code, math, figures, links, and JSON-LD structure.CI and contributor workflow: Unit tests and existing suites (
blog-content,i18n,zh-copy) call into the new module; mutation tests cover each rule. PRs labeledchinese-copy-onlytrigger a separatezh-copy-onlyworkflow job that runsguard:zh-copy:chinese-onlyagainst the merge base to block accidental English-byte changes (full English MDX files and orderedeninitializer subtrees). Label add/remove events are wired so that job can run without re-running the full vitest job on label-only events.Docs and deps:
AGENTS.mdanddocs/chinese-copy.mddocument the objective vs subjective boundary.@mdx-js/mdxis added for blog structure parsing.CollectiveXDisplaytrims extrazhdictionary keys so they align with the English shape under the new parity rule.Reviewed by Cursor Bugbot for commit 484dcf8. Bugbot is set up for automated code reviews on this repo. Configure here.